Present invention relates to an image data processing apparatus which enlarges the dynamic range of image data to be processed.
Recently, solid-state image sensing devices such as a CCD image sensor are employed as image pick-up devices. However, as the dynamic range of the solid-state image sensing device is smaller than that of a device using silver chloride, image quality may be degraded depending upon image-sensing conditions.
One dynamic-range enlarging method is obtaining a plurality of image data from one scene by sensing with different amount exposure, and combining the plurality of image data by some calculation to obtain combined image data with an enlarged dynamic range. The obtained image data may be outputted by an output device such as a display monitor.
Japanese Patent Application Laid-Open No. 60-52171 discloses combining a plurality of image data obtained with different exposure amounts by adding image signals as shown in FIGS. 1A and 1B. The figures show how dynamic range is enlarged by addition of images signals.
In FIGS. 1A and 1B, the horizontal axis represents the luminance of the object; the vertical axis, the output of the image sensing device; the solid line a, the input-output characteristic of the image sensing device when the exposure amount is increased; and the dotted line b, the input-output characteristic when the exposure amount is decreased. Adding the respective output values in FIG. 1A produces output of enlarged dynamic range as shown in FIG. 1B.
Japanese Patent Application Laid-Open No. 63-306777 discloses cutting out partial image data and combining the cut-out data and another image data to obtain combined image data as shown in FIGS. 2A to 2C. The figures illustrate how cutting off and combining images are made. As shown in FIG. 2A, if the object scene has a wide dynamic range, image sensing with standard exposure obtains image data where the portion of the person (main object) is darkened. Then, the prior art senses the same scene with increased exposure, (the background is whitened), and the main object portion is cut out from the obtained image data, as shown in FIG. 2B. Combining the two image data as shown in FIGS. 2A and 2B provides image data with an enlarged dynamic range as shown in FIG. 2C.
However, these conventional techniques have various problems. First, upon outputting image data having enlarged dynamic range on an output device such as a display monitor or a printer, the dynamic range of the output device may not correspond to that of the combined image. In this case, appropriate compressing operation is performed so that the dynamic range of the combined image is adjusted in accordance with the dynamic range and output characteristic of the output device. However, the improper compressing degrades the image contrast or the balance of brightness, thus provides a very unnatural image, compared with a standard image taken with proper exposure before combining processing. Further, a number of times of compressing and evaluation of compressed image are to be repeated until a good image is obtained.
FIGS. 3A and 3B illustrate a problem raised in the addition of image signals according to Japanese Patent Application Laid-Open No. 60-52171 in FIGS. 1A and 1B. FIGS. 3A and 3B illustrate how brightness increases due to in intermediate bright area. Image sensing elements according to the prior art do not form a continuous sensing area for increased and decreased exposed images, and further the characteristics of the exposures are not equal. Consequently, as shown in FIG. 3B, the result of addition of image signals where the area of intermediate brightness has increased luminance. That is, the image has comparatively high luminance and lacks contrast.
In cutting out and combining of image data according to Japanese Patent Application Laid-Open No. 63-306777, where a dark-corrupted area of standard image data obtained from sensing with proper exposure (FIG. 2A) is replaced with a corresponding area of another image data obtained from sensing with different exposure (FIG. 2B), the combined area yields pseudo outlines due to different luminance levels of the both image data, i.e., the difference between output characteristics of the image sensing device with respect to the object and the luminance level of the background image data. If the luminance levels of the respective images are adjusted to the same level to prevent occurrence of pseudo outlines, the enlargement of dynamic range cannot be fully obtained.
Further, in the conventional image sensing device, upon sensing a moving object or applying sensed image data to a movie, a plurality of image data sensed with different exposures are combined. If a single image sensing device is used for obtaining the plurality of image signals, a time difference between the initial image which is first picked up and the subsequent images picked up later causes loss of simultaneousness in the respective combined images. In order to perform high-speed reading to reduce the time difference between plurality of images, the image sensing device will need a complicated driving mechanism. For these reasons, in a case where a plurality of image sensing devices are used to reduce the time difference between a plurality of images, an ND filter may be used for changing exposure. However, using ND filter upon sensing one scene needs a fixed exposure change, otherwise, the exposure cannot be freely changed in accordance with the illuminance upon the object or the movement of the object. Further, a complicated attachment/detachment mechanism is required for ND filter.
Conventionally, methods for sensing an object and combining a plurality of image data into one image data for one frame have been introduced, however, none has suggested as to preparing a plurality of image data to be combined to obtain high-quality image data with an enlarged dynamic range.